.xgp
.FONT 0 25VG
.FONT 1 25VRI
.FONT 2 S25GRK
.FONT 3 37VRB
.adjust
.sp 3
.twinch 7.0
.center
ε3FROM UNDERSTANDING COMPUTATION TO 
.sp
.center
UNDERSTANDING NEURAL CIRCUITRY
.sp 6
.center
by
.sp 3
.center
D. Marr and T. Poggioε0*
.sp 6
.he1
Marr & Poggio
.he2
From computation to neural circuitry
.double
.center
The Artificial Intelligence Laboratory,
.br
.center
Massachusetts Institute of Technology,
.br
.center
545, Technology Square,
.br
.center
Cambridge, Mass. 02139, U.S.A.
.sp 2
.twinch 6.0
* Permanent address: Max-Planck Institut fur Biologische Kybernetik,
.br
74 Tubingen 1, Spemannstrasse 38, Germany.
.sp 2
Running title: From computation to neural circuitry
.pa
.center
ε1Summaryε0
	The CNS needs to be understood at four nearly independent levels
of description: (1) that at which the nature of a computation is expressed;
(2) that at which the algorithms that implement a computation are characterised;
(3) that at which an algorithm is committed to particular mechanisms; and (4)
that at which the mechanisms are realised in hardware. In general, the nature of
a computation is determined by the problem to be solved, the mechanisms that
are used depend upon the available hardware, and the particular algorithms chosen
depend on the problem and on the available mechanisms. Examples are given of
theories at each level.
.pa
.center
ε1FROM UNDERSTANDING COMPUTATION TO UNDERSTANDING NEURAL CIRCUITRYε0
	Modern neurophysiology has learned much about the operation
of the individual neuron, but deceptively little about the meaning
of the circuits they compose.  The reason for this can be attributed, at
least in part, to a failure to recognise what it means to understand a
complex information-processing system.
	Complex systems cannot be understood as a simple extrapolation of the
properties of their elementary components.
One does not formulate a description of thermodynamical
effects using a large set of wave equations, one for each of the particles involved.
One describes such effects at their own level, and tries to show that in
principle, the microscopic and macroscopic descriptions are consistent with
one another.
	The core of the problem is that a system as complex as a
nervous system or a developing embryo must be analyzed and understood
at several different levels.  For a system that solves an
information processing problem, we may distinguish four important levels of
description.  At the lowest, there is basic component and circuit
analysis -- how do transistors, neurons, diodes and synapses work?  The
second level is the study of particular mechanisms; adders, multipliers,
and memories accessed by address or by content.  The third level is that
of the algorithm, and the top level contains the theory of the overall
computation.  For example, take the case of Fourier analysis.  The computational
theory of the Fourier transform is well understood, and is expressed
independently of the particular way in which it is computed.  One level
down, there are several algorithms for implementing a Fourier transform
-- the Fast Fourier transform (Cooley & Tukey 1965) which is a serial
algorithm; and the parallel "spatial" algorithm that is based on the
mechanisms of laser optics.  All these algorithms carry out the same
computation, and the choice of which one to use depends upon the
particular mechanisms that are available.  If one has fast digital
memory, adders and multipliers, one will use the FFT, and if one has
a laser and photographic plates, one will use an "optical" algorithm.  In
general, mechanisms are strongly determined by hardware, the nature of
the computation is determined by the problem, and the algorithms are
determined by the computation and the available mechanisms.
	Each of these four levels of description has its place in the eventual
understanding of perceptual information processing, and it is important
to keep them separate.  Of course, there are are logical and causal relationships
among them, but the important point is that these levels of description are
only loosely related.  Too often in attempts to relate psychophysical
problems to physiology there is confusion about the level at which a
problem arises - is it related mainly to biophysics (like after-images)
or primarily to information processing (like the ambiguity of the Necker
cube)? More disturbingly, although the top level is the most neglected,
it is also the most important.  This is because the structure of the
computations that underly perception depend more upon the computational
ε1problemsε0 that have to be solved than on the particular hardware in
which their solutions are implemented.  There is an analog of this in
physics, where a thermodynamical approach represented, at
least historically, the first stage in the study of matter.  A
description in terms of mechanisms or elementary components usually appears
afterwards.
	Our main point then is that the topmost of our four levels, that
at which the necessary structure of computation is defined, is a crucial
but neglected one.  Its study is separate from the study of particular
algorithms, mechanisms or hardware, and the techniques needed to pursue
it are new.  In the rest of the article, we summarize some examples of vision
theories at the different levels we described, and illustrate the types
of prediction that can emerge from each.
.br
.center
(figure 1 near here)
.sp
.center
ε1Examples of computational theoriesε0
.br
ε11: Orientation behavior of the flyε0 (Reichardt, 1970; Reichardt &
Poggio, 1976)
	The flight behavior of houseflies requires an elaborate 
visual flight control system. Houseflies perceive motion relative
to the environment and thereby  stabilize their flight course; they locate
and fly towards prominent objects; they are able to track moving targets and
to chase other flies; they discriminate or prefer some specific visual patterns.
Work at the Max-Planck-Institut over the last few years has provided a
good understanding of part of this control system. We refer
here especially to the approach of Reichardt and Poggio (1976), which represents an
example of a theory at the fourth, computational level.
The overall computation is defined and well accessible to
experimentation, since it involves a complete input-output transduction,
from the optical input to the behavioral motor output. The theoretical
description and most of the related experiments are as yet restricted to
a specific part of the orientation behavior of flies.  However, the theory
accounts in a quantitative way for orientation, chasing and spontaneous
pattern preference behaviour, at least in either one of two degrees of
dynamical freedom.  Although connections with physiological and
anatomical data are being established, the theory is based on behavioral
data.  The theory leads to the following equation, which quantitatively
describes and "predicts" the angular trajectory ε2yε0(t) of a fly
fixating or tracking an object moving with angular speed  ε2yε0(t):
.br 
.center
ε2Qyε0(t) + ε0kε2yε0(t) + kε2wε0(t) = -D[ε2yε0(t)] - rε2yε0(t) + N(t)          (1)
.br
where the angular error ε2yε0(t) represents the instantaneous position of
the pattern on the retina of the fly.  The terms on the
left hand side represent the flight dynamics (ε2Qε0 is the moment of
inertia of the fly, k is a rotational friction constant).  ε2wε0(t) is the
angular speed of the object. The right hand side describes the
instantaneous torque of the fly; the term N(t) is a zero-mean random
process, and is independent of the visual input;  rε2yε0(t), a velocity-dependent
optomotor response, is the result of a "movement computation";  D[ε2yε0]
carries the position information, acquired from the visual input by a
"position computation" and represents the "attractiveness" profile
associated with the specific pattern. All these terms have been characterized
quantitatively, through independent experiments.  For instance, D[ε2yε0(t)]
can be measured as the mean torque generated by a fixed, flying fly when the 
image of the given object is flickered (to avoid a stabilized image) on the
retina at position ε2yε0.
	Through equation (1) (and natural extensions of it) the theory predicts
a rather complex  behavior: fixation of several different patterns,
various instances of tracking and some simple cases of pattern
"discrimination" (Poggio and Reichardt, 1973; Reichardt and Poggio,
1975).  Figure 1 gives two examples of behavior that is quantitatively
explained by this approach.  In summary the theory outlines the basic
logical organization of the visual control system of the fly. It holds
that the nervous system performs two main computations on the visual
input, one extracting movement infomation (the term rε2yε0(t)), the
other providing position infomation (the term D[ε2yε0],and that these
two terms determine the (closed-loop) behaviour described by equation 1.
	There is an approximate rule for determining the D[ε2yε0]
and the r that are associated with a given
pattern. The rule states that the "attractiveness" D[ε2yε0]
of a pattern that can be decomposed into several vertical edges or stripe
segments, can be derived, to a first approximation, from the linear superposition
of the "attractiveness" profiles of each component. The
precise justification for this rule and its range of validity must depend on
the neural interactions that compute D[ε2yε0] and rε2yε0. A later
section describes briefly this level of analysis (the "algorithm"
level).
	A general remark is worth making here.
The quantitative description of equation (1) could
not have been obtained from single cell recordings or from histology.
This is a rather clear example where there seems to be little predictive
extrapolation from the "component" level to the "computational" level.
Extrapolation in the other direction is, however, somewhat easier:
equation 1 is probably a prerequisite of any full understanding at the
level of circuitry.  For instance, it provides a few suggestions about
the modular organization of the underlying nervous network, and several
criteria for an physiological localization of functionally separated
modules (Reichardt and Poggio, 1976).
.sp
.center
ε1More complex visual systemsε0
	Since the early 1950's, there has been considerable progress in the
study of vertebrate and especially mammalian visual systems.  The technology of
single-cell recording initiated what is widely regarded as a breakthrough in
visual neurophysiology (Hubel & Wiesel 1962).
The use of the computer in psychphysics allowed
B. Julesz (1971) to construct random-dot stereograms, and very recently it made it
possible for R. N. Shepard (1975) and his collaborators to explore with precision
the phenomena of "mental rotation".
	Interesting and important though these findings are, one must
sometimes be allowed the luxury of pausing to reflect upon the overall
trends that they represent, in order to take stock of the kind of
knowledge that is accessible to these techniques.  This bulletin is itself an
attempt at examining the link between the two current approaches, neurophysiology
and psychophysics.  We would also like to know what
are the limitations of these approaches, and how can one compensate for their
deficiencies?
	Perhaps the most striking feature of these disciplines at present is their
phenomenological character.  They describe the behavior of cells or of
subjects, but do not explain it. What is area 17 actually doing? 
What are the problems in doing it that need explaining, and at
what level of description should such explanations be sought?
	In trying to come to grips with these problems, Marr has adopted a
point of view that regards visual perception as
a problem primarily in information processing. The problem commences with
a large, gray-level intensity array, and it culminates in a ε1descriptionε0
that depends on that array, and on the purpose that the viewer brings to it.
Viewed in this light, a theory of visual information processing
will exhibit the four levels of description that, as we saw in the
introduction, are attached to any device that solves an information processing
problem; and the first task of a theory of vision is to examine the top
level. What exactly is the underlying ε1natureε0 of the computations
being performed during visual perception?
	The empirical findings of the last 20 years, together with
related anatomical (Allman 1972, 1973, 1974a, b & c,
Zeki 1971) and clinical (e.g. Luria 1970, Critchley
1953, Vinken & Bruyn 1969) experience, have
strengthened a view for which widespread indirect evidence previously
existed, namely that the cerebral cortex is divided into many different
areas that are distinguished structurally, functionally and by their
anatomical connections. This suggests that, to a first approximation
visual information processing can be thought of as having a ε1modularε0
structure, a view which is strongly supported by evolutionary arguments.
If this is true, the task of a top-level theory of vision is clear; what are the
modules, what does each do, and how?
	The approach of the M. I. T. Artificial Intelligence Laboratory
to the vision problem rests on these assumptions.
We believe that the principal problems at present are (a) to formulate
the likely modularization, and (b) to understand the computational
problems each module presents.  Unlike the case of the fly, the first step is the
most difficult, just as formulating a problem in physics is often more
difficult for a skilled mathematician than solving it.
Nevertheless a variety of clues is available, from psychophysics and
neurohysiology to the wide and interesting range of deficits reported in
the literature of clinical neurology. Those cases in which a patient
lacks a particular, highly circumscribed faculty are particularly
interesting (e.g. Warrington & Taylor 1973, Efron 1968); but more
general impairments can also be informative, particularly the agnosias in
which higher level analysis and interpretation are damaged while leaving
other functions, like texture discrimination and figure-ground
separation, relatively unimpaired. Such evidence must be treated with
due caution, but it encourages us to examine ways of squeezing the last
ounce of information from an image before taking recourse to the
descending influence of high-level interpretation on early processing.
Computational evidence can also be useful in suggesting that a certain
module may exist. For example, Ullman (1975) showed that fluorescence
may often be detected in an image using only local cues, and the method
is so simple that one would expect something like it to be incorporated
as a computational module in the visual system, even though we were not
aware of any supporting evidence, either clinical, physiological, or
psychological. The same may be true of other visual qualities like
glister and wetness, just as it is generally believed to be true for
color, motion and stereopsis.
	In order to introduce the reader to our approach, the next few
sections present brief summaries of a particular modularization, and
the associated theories, that we have been studying
over the last two years. We are perfectly aware
that the particular decomposition chosen here may not be exactly correct,
and even if it is, the separation of modules is certainly not complete.
All of the modules described here have been implemented in computer programs
which demonstrate that this particular scheme works for a number of
natural images. Alternative decompositions that have been tried, in
particular those that rely on much more interaction between low-level
processing and high-level interpretation of an image, (e.g. Freuder (1975),
Shirai (1973)) have not hitherto led to such satisfactory and promising results.

.block 4
ε11: The Primal Sketchε0 (Marr 1976a)
	It is a commonplace that a scene and a drawing
of the scene appear very similar, despite the completely different grey-level images to
which they give rise.  This suggests that the artist's local symbols correspond in some
way to natural symbols that are computed out of the image during the normal course
of its interpretation.  The first part of our visual information
theory therefore asserts that the first operation on an image
is to transform it into its raw ε1primal
sketchε0, which is a primitive but rich ε1descriptionε0 of the intensity changes
that are present. Figure 3 shows an example. In order to obtain this description,
approximations to the first and second directional derivatives of intensity are
measured at several orientations and on several scales everywhere in the image,
and these measurements are combined to form local
descriptive assertions.  The process of computing the primal sketch
involves five important steps, the first of which can be compared with the
measurements that are apparently made by simple cells in the visual cortex. 
One prediction made by this part of the theory
is that a given intensity change itself determines which simple-cell
measurements are used to describe it.  This is in direct contrast to theories which
assert that each simple cell acts as a "feature-detector", whose output is freely
available to subsequent processes.  If this is true, it requires that
a well-defined interaction take place between simple-cell like measurements
made at the same orientation and position in the visual field but with different
receptive field sizes (see Marr 1976a).
.center
(figure 2 near here)
.br
ε12: Stereopsisε0 (Marr 1974, Marr & Poggio 1976)
	Suppose that images of a scene are available, taken from two
nearby points at the same horizontal level. In order to compute stereoscopic
disparity, the following steps must be carried out: (1) a particular
location on a surface in the scene must be chosen from one image; (2) that
location must be identified in the other image; (3) the relative positions
of the two images of that location must be measured. Notice that methods based on
grey-level correlation between images fail to satisfy these conditions
because a grey-level measurement does not define a point on a physical surface
independently of the optics of the imaging device.
The matching must be based on objective markings that lie on a physical surface,
and so one has to use predicates that correspond to changes in reflectance.
One way of doing this is to obtain a primitive description of the intensity changes
that exist in each image, and then to match these descriptions.
Line and edge segments, blobs, and edge termination points correspond quite closely
to boundaries and reflectance changes on physical surfaces.
	The stereo problem may thus be reduced to that of matching two primitive
descriptions, one from each eye. One can think of elements of these
descriptions as having only position information, like the black points
in a random-dot stereogram, although in practise there exist some rules about
which matches between descriptive elements are possible, and which are not.
There are physical constraints that translate into two rules for how the left and right
descriptions are combined:
.br
(1) ε1The uniqueness conditionε0. Each item from each image may be assigned
at most one disparity value. This condition rests on the premise that the
items to be matched have a physical existence, and can be in only one place at a time.
.br
(2) ε1Continuityε0. Disparity varies smoothly almost everywhere. This condition
is a consequence of the cohesiveness of matter, and it states that only a relatively
small fraction of the area of an image is composed of boundaries.
.br
These conditions  on the computation are represented geometrically
in figure 2a. Later in the article, we exhibit a network that implements
these conditions, and we illustrate how it solves random-dot stereograms.
	In this case the computational problem is rather well-defined, essentially
because of Julesz's (1971) demonstration that random-dot stereograms, containing
no monocular information, yield stereopsis.  It is not yet completely clear however
what mechanisms are actually available for implementing this computation (for instance
do eye movements play a critical role?).  As a consequence, it is an open question
whether the cooperative algorithm introduced later is used, or whether simpler
"serial" scanning algorithms may actually be implementing the stereopsis
computation (see Marr & Poggio 1976).
.br
.center
(figure 3 near here)
.br
ε13: Grouping and texture visionε0 (Marr 1976a)
	The primal sketch of an image is in general a large and
unwieldy collection of data.  This is an unavoidable
consequence of the irregularity and complexity of natural images. 
The next important computational problem is how to decode the primal sketch. 
For most images, it appears unnecessary to invoke specific hypotheses about what is there
until considerably later in the processing.  The theory next applies
a number of quite general selection and grouping processes to elements in
the primal sketch.  The purpose of these processes is to organise the local
descriptive elements into ε1formsε0 and ε1regionsε0, which are
closed contour groups that are obtained in various ways.
Regions may be defined by their boundaries, which have been formed by
grouping together some set of edge, line, or place-tokens; or they may be defined
by a first-order predicate operating on the primal sketch elements within
it. This second method corresponds to the definition of a region by a texture,
and it leads to a theory of the processes on which texture discrimination is
based.
	It is important to realize that
the descriptive items that may be grouped here can be very abstract - like
tokens for the end of a line, a blob, or a constructed line that joins two blobs. 
Tokens are created for each new group, and these
tokens themselves become subject to the operation of the same or similar
grouping processes as operated on elements of the raw primal sketch. 
The grouping processes are very conservative.  They satisfy a principle that seems
to have general application to recognition problems, called the ε1principle
of least commitmentε0, which states that nothing should be done that may
later have to be undone.  Only "obvious" groupings are made,
and where there is doubt between two possible groupings, both are constructed
and held pending subsequent selection. Figure 3 illustrates some results of
applying these grouping processes. 
.sp
ε14: 3-D representation of shapeε0 (Marr & Nishihara 1975)
	The last two components of the theory concern the representation
of three-dimensional shapes.  One component deals with the nature of the
representation system that is used, and the other with how to obtain it
from the types of description that can be delivered from the primal sketch.
The key ingredients of the representation system are:
.br
(a) The deep structure of the three-dimensional representation of an
object consists of a stick figure, where in formal terms each
stick represents one or more axes in the object's generalized cone
representation, as illustrated in figure 4f.
In fact, a hierarchy of stick figures exists, that allows
one to describe an object on various scales with varying degrees of detail.
.br
(b) Each stick figure is defined by a propositional database called a ε13-D model.ε0
The geometrical structure of a 3-D model is specified by storing the
relative orientations of pairs of connecting axes.  This specification is
local rather than global, and it contrasts with schemes in which the position of
each axis is specified in isolation, using some circumscribing frame of reference.
.br
(c) When a 3-D model is being used to interpret an image, the geometrical
relationships in the model are interpreted by a computationally simple
mechanism called the ε1image-space processorε0, which may be thought of as a
device for representing the positions of two vectors in 3-space, and for
computing their projections onto the image.
.br
(d) During recognition, a sophisticated interaction takes place between
the image, the 3-D model, and the image-space processor.  This interaction gradually
relaxes the stored 3-D model onto the axes computed from the image. 
Some facets of this process resemble
the computation of a 3-D rotation, but a simple computer graphics metaphor
is misleading. In fact, the rotations take place on abstract vectors (the axes)
that are not even present in the original image; and at any moment, only
two such vectors are explicitly represented.
	The essence of this part of the theory is a method for
representing the spatial disposition
of the parts of an object and their relation to the viewer.
.br
.center
(figure 4 near here)
.br
ε15: 2 1/2 - dimensional analysis of an imageε0 (Marr 1976c, Marr & Vatan, in preparation)
	In simple images, the forms delivered from the primal sketch
correspond to the contours of physical objects.
Finally therefore, we need to bridge the gap between such forms and
the beginning of the 3-D analysis described in the previous paragraph.
We call this 2 1/2 - 
dimensional analysis, and it consists largely of assigning to contours labels,
that reflect aspects of their 3-dimensional configuration, before that configuration
has been made explicit.  The most powerful single idea here is the distinction between
convex and concave edges and contour segments.  One can show that these distinctions
are preserved by orthogonal projections, and can be made the basis of a segmenting
technique that decomposes a figure into 2-D regions that correspond to the
appropriate 3-D decomposition for a wide range of viewing angles (see figure 4).  
Marr (1976c) has proved that the assumptions, that are implicit in the use of
the convex-concave distinction to analyze a contour, are equivalent to assuming
that the viewed shapes are composed of generalized cones. This adds additional
support for using the stick-figure scheme based on generalized cones to
represent 3-D shapes.
The theory assigns many alternating figure effects like the Necker cube to
the existence of alternative self-consistent labellings computed at this stage.
	It is perhaps worth mentioning one interesting point that has emerged from this
way of recognising and representing 3-D shapes.  Warrington & Taylor (1973)
described patients with right parietal lesions who had difficulty in recognising
objects seen in "unconventional" views - like the view of a water pail seen from
above.  They did not attempt to define what makes a view unconventional.  According
to our theory, the most troublesome views of an object will be those in which
its stick-figure axes cannot easily be recovered from the image. 
The theory therefore predicts that
unconventional views in the Warrington & Taylor sense will correspond to those
views in which an important axis in the object's generalised cylinder
representation is foreshortened.  Such views are by no means uncommon - if a 35mm camera
is directed towards you, you are seeing an unconventional view of it, since the axis
of its lens is foreshortened.

.block 4
.center
ε1Examples of algorithms and mechanismsε0
	Between the top and bottom of our four levels lie descriptions of
algorithms and descriptions of mechanisms.  The distinction between these two
levels is rather subtle, since they are often closely related.  The
form of a specific algorithm can impose strong constraints on the
mechanisms,  and conversely.   Let us consider three examples.
.br
.center
(figure 5 near here)
.br
ε11: "Simple" algorithmsε0
	An algorithm operates on some kind of input and yields a
corresponding output.  In formal terms, an algorithm can be thought of
as a mapping between the input and the output space. Perhaps the
simplest of all nonlinear operators on a linear space are the so-called
polynomial operators.  They encompass a broad spectrum of applications
including all linear problems, and they approximate all sufficiently smooth,
nonlinear operators.  For this particular class of "simple" algorithms
(i.e. representable through a "smooth" operator) polynomial
representations provide a canonical decomposition in a series of
simpler, multilinear operators. Figure 5 shows this decomposition in
terms of interactions or "graphs" of various orders: in this way an algorithm, or
its network implementation, may be decomposed into an additive sequence of
simple, canonical terms, just as in another context, a function can be conveniently
characterized by its various Fourier terms.   Moreover, functional and
computational properties can  be associated with interactions of a given
order and type.
	Poggio & Reichardt (1976) used the polynomial representation of functionals
to classify the algorithms underlying movement, position
and figure-ground computation in the fly's visual system. The idea was to
identify which terms, among the diversity of the possible ones, are
implied by the experimental data.  Figure 5  shows the graphs that play a
significant role in the fly's control of flight and, in this sense,
characterize the algorithms involved.  The notion that seems to capture
best the  "computational complexity" of these simple, smooth mappings is
the notion of p-order (perceptron-order, see Poggio and Reichardt, 1976). 
Movement computation in the fly is of order 2, and figure-ground
discrimination in the simple case of relative motion depends on
fourth-order graphs, but possibly with p-order 2.  A closed or Type 1
(Marr 1976b) theory of this kind may be a useful way of characterizing
preprocessing operations in nervous systems.  The approach has a rather
limited validity however, since it does not apply to the large and
important class of "non-smooth" algorithms, where cooperative effects,
decisions and symbols play an essential role. While an arbitrary  number
of mechanisms and circuits may implement these "smooth" algorithms, it
is clear that "forward" interactions between neurons are the most
natural candidates.
	Although the various levels of description are only loosely related,
knowledge of the computation and of the algorithm may sometimes admit
inferences at the lowest level of anatomy and physiology. The
description of the visual system of the fly at the computational and
functional level suggests, for instance, that different, separate neural
structures may correspond to the different computations. Recent data
support this conjecture. Movement computation (the term rε2yε0(t) of equation
(1) and the second order graph of Figure 5) seems to depend mainly on
receptor system 1-6, while the position computation (the term D(ε2yε0(t))
of equation (1) and the "self-graph" of Figure 5) seems dependent on receptor
system 7-8 (Wehrhahn, 1976 and in preparation). Mutants of Drosophila,
normal with respect to the movement algorithm, are apparently disturbed
in the position algorithm (Heisenberg, in preparation).
.sp
ε12: "Cooperative" algorithmsε0
	A more general and not precisely definable class of algorithms
includes what one might call cooperative algorithms. Such algorithms
may describe bifurcations and phase transitions in dynamical systems. An
essential feature of a cooperative algorithm is that it operates on many
"input" elements and reaches a global organization ε1viaε0 local but highly
interactive constraints.  An apparently cooperative algorithm plays a
major role in binocular depth perception (Julesz 1971).  The
stereopsis computation defined by figure 2a  applies many
local constraints to many local inputs to yield a final state consistent
with these constraints.  Various mechanisms could implement this type of
algorithms. Parallel, recurrent, nonlinear interactions, both excitatory
and inhibitory, seem to represent a natural impementation.  In the
stereopsis case such a mechanism is illustrated in the rest of figure 2.  This mechanism
may be realized through many different components and circuitries.  In
the nervous system, however, there are certain very obvious candidates,
which allow some definite predictions.  For instance, one is led
to conjecture the existence of disparity columns (actually layers)
of cells with reciprocal excitatory short-range interactions on each
layer and long-range inhibitory interactions between layers with the
characteristic "orthogonal" geometry of figure 2. Figure 6 shows that
this algorithm succesfully extracts depth information from random-dot
stereograms. The algorithm exhibits typical cooperative phenomena, like
hysteresis and disorder-order transitions.   It is important to stress
that it is the computational problem which determines the structure of
the excitatory and inhibitory interactions, and not "hardware"
considerations about neurons or synapses.  The apparent success of this
cooperative algorithm in tackling the stereo problem suggests that other
perceptual computations may be easy to implement in similar ways.
Likely candidates are "filling-in" phenomena, subjective
contours, figural reinforcement,  some kinds of perceptual grouping and
associative retrieval. In fact the associative retrieval network described by
Marr (1971), in connexion with a theory of the hippocampal cortex, implements a
cooperative algorithm.
.br
.center
(figure 6 near here)
.br
ε13: Procedural algorithmsε0
	Still another and larger class of algorithms is represented by
the specification of procedures, and the construction and
manipulation of explicit symbolic descriptions. For example, the 3-D
representation theory described in part 4 of the previous section explains how the
stick figure representation of a viewed object may be obtained from an image, and
manipulated during recognition. The detailed specification of the
algorithms involved here is carried out by defining the datastructures
that are created to represent the situation, and by specifying
procedures that operate on these datastructures in accordance with the
information currently being delivered from the image, and that available
from stored models.
	This way of specifying an algorithm is very general and
powerful, although unlike the two other ways that we discussed, it is a
far cry from the circuitry level of description at which
neurophysiological experiments are carried out. In a digital computer,
one does not try to bridge the gap between these two levels in one step.
Instead, a basic instruction set, an assembler, a high level language (LISP,
ALGOL) and a compiler are interposed to ease the burden of passing from the
description of a computation down to the specification of a particular
pattern of current flow.
	We may eventually expect a similar intermediate vocabulary to be
developed for describing the central nervous system. Hitherto, only one non-trivial
"machine-code" operation has been studied in the context of neural
hardware, namely simple storage and retrieval functions (Marr 1969 & 1971, and
Brindley 1969).
.sp
.single
ACKNOWLEDGEMENTS: This report describes research done at the Artificial
Intelligence Laboratory of the Massachusetts Institute of Technology. Support
for the laboratory's artificial intelligence research is provided in part by the
Advanced Research Projects Agency of the Department of Defense under Office
of Naval Research contract N00014-75-C-0643. T. P. acknowledges the financial
support of the Max-Planck-Gesellschaft during his visit to M. I. T.
.sp 2
.center
ε1Referencesε0

Allman, J. M., Kaas, J. H., Lane, R. H. & Miezin, F. M. (1972)  A representation
of the visual field in the inferior nucleus of the pulvinar in the owl monkey
ε1(Aotus trivirgatus). Brain Research, 40ε0, 291-302.

Allman, J. M., Kaas, J. H. & Lane, R. H. (1973)  The middle temporal visual
area (MT) in the bushbaby, ε1Galago senegalensis. Brain Research, 57ε0, 197-202.

Allman, J. M. & Kaas, J. H. (1974a)  The organization of the second visual area
(V-II) in the owl monkey: a second order transformation of the visual hemifield.
ε1Brain Research, 76ε0, 247-265.

Allman, J. M. & Kaas, J. H. (1974b)  A visual area adjoining the second visual
area (V-II) on the medial wall of parieto-occipital cortex of the owl monkey
ε1(Aotus trivirgatus). Anat. Rec., 178ε0, 297-8.

Allman, J. M. & Kaas, J. H. (1974c)  A crescent-shaped cortical visual area
surrounding the middle temporal area (MT) in the owl monkey ε1(Aotus trivirgatus).
Brain Research, 81ε0, 199-213.

Brindley, G. S. (1969). Nerve net models of plausible size that perform many
simple learning tasks. ε1Proc. Roy. Soc. B., 174ε0, 173-191.

Cooley, J. M. & Tukey, J. W. (1965). An algorithm for the machine computation
of complex Fourier series. ε1Math. Comp., 19ε0, 297-301.

Critchley, M. (1953)  ε1The parietal lobesε0. London: Edward Arnold.

Eforn, R. (1968). What is perception? In:ε1Boston studies in the philosophy
of science, IVε0, Eds. R. S. Cohen & M. W. Wartovsky. Dordrecht, Holland: Reide.

Freuder, E. C. (1975). A computer vision system for visual recognition using
active knowledge. ε1M. I. T. A. I. Lab. Technical Report 345ε0.

Geiger, G. & Poggio, T. (1975). The Mueller-Lyer figures and the fly. ε1Science
 190ε0, 479-480.

Hubel, D. H. & Wiesel, T. N. (1962). Receptive fields, binocular interaction and
functional architecture in the cat's visual cortex. ε1J. Physiol. (Lond.),
160ε0, 106-154.

Julesz, B. (1971).  ε1Foundations of Cyclopean Perceptionε0. Chicago: The University
of Chicago Press.

Marr, D. (1969). A theory of cerebellar cortex. ε1J. Physiol. (Lond.) 202ε0, 437-470.

Marr, D. (1971). Simple memory: a theory for archicortex.
ε1Phil. Trans. Roy. Soc. B., 252ε0, 23-81.

Marr, D. (1974). A note on the computation of binocular disparity in a
symbolic, low-level visual processor. ε1M. I. T. A. I. Lab. Memo 327.ε0

Marr, D. (1976a). Early processing of visual information.
ε1Phil. Trans. Roy. Soc. B.ε0 (in the press).

Marr, D. (1976b). Artificial Intelligence -- a personal view.
ε1M. I. T. A. I. Lab. Memo 355.ε0

Marr, D. (1976c). Analysis of contour. ε1M. I. T. A. I. Lab.
Memo 372.ε0

Marr, D. & Nishihara, H. K. (1975). Spatial disposition of axes in a generalized
cylinder representation of objects that do not encompass the viewer.
ε1M. I. T. A. I. Lab. Memo 341.ε0

Marr, D. & Poggio, T. (1976). Cooperative computation of stereo disparity.
ε1Scienceε0, (submitted for publication).

Poggio, T. & Reichardt, W. (1973). A theory of the pattern induced flight
orientation of the fly ε1Musca Domesticaε0.  ε1Kybernetik 12,ε0185-203.
 
Poggio, T. & Reichardt, W. (1976). Visual control of the orientation behavior of
the fly: towards the underlying neural interactions.  ε1Quarterly Reviews in
Biophysicsε0, (in the press).

Reichardt, W. (1970). The insect eye as a model for analysis of uptake, transduction
and processing of optical data in the nervous system. In:ε1 The Neurosciences, Second
Study Programε0. (ed. F. O. Schmitt), 494-511. New York: The Rockfeller University
Press.

Reichardt, W. & Poggio, T. (1975). A theory of pattern induced flight orientation
of the fly ε1Musca Domesticaε0 II. ε1Kybernetik, 18,ε069-80.

Reichardt, W. & Poggio, T. (1976). Visual control of the orientation behavior of 
the fly: a quantitative analysis. ε1Quarterly Reviews in Biophysicsε0, (in the press).

Shepard, R. N. (1975). Form, formation, and transformation of internal
representations. In: ε1Information processing and cognition: The Loyola
Symposiumε0, Ed. R. Solso, pp 87-122. Hillsdale, N. J.: Lawrence Erlbaum Assoc.

Shirai, Y. (1973). A context-sensitive line finder for recognition of polyhedra.
ε1Artificial Intelligence, 4ε0, 95-120.

Vinken, P. J. & Bruyn, G. W. (1969)  Eds. ε1Handbook of Clinical Neurology: Vol. 2,
Localization in Clinical Neurologyε0. (In association with A. Biemond).
Amsterdam: North Holland Publishing Co.

Warrington, E. K. & Taylor, A. M. (1973).  The contribution of the right parietal
lobe to object recognition. ε1Cortex, 9ε0, 152-164.
 
Wehrhahn, C. (1976). Evidence for the role of retinal receptors R 7/8 in the 
orientation behavior of the fly.  ε1Biol. Cybernetics, 21ε0, 213-220.

Zeki, S. M. (1971)  Cortical projections from two prestriate areas in the
monkey. ε1Brain Research, 34ε0, 19-35.
.end